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Placement Testing and Student Success: 
The First Intervening Variable 

Alan J. Sturtz 

Director of Institutional Research, Planning and Development 
Gateway Community-Technical College 
and 

Judith A. McCarroU 
Director of Computer Services 
Naugatuck Valley Community-Technical College 

Introduction 

Community colleges have long provided pre-coUege courses (alternatively called 
remedial and developmental) for those students who do not possess the academic skills- in 
English and Mathematics to succeed in college-level courses. However, in current times of 
dwindling resources and increased institutional accountability, many colleges are requiring 
their students to sit for various types of placement tests, either locally developed or state or 
national standardized examinations. This serves the two-fold purpose of directing students to 
courseb where their chance to succeed is greater than their chance to fail and making the best 
use of personnel resources for both pre-college and college-level courses. 

In Connecticut, the Board of Trustees for Community-Technical Colleges, "in 
recognition of the need to extend to an increasingly diverse student clientele access to 
educational opportunity and the need, to provide a wide variety of instructional and student 
support services to assist students achieve their objectives," mandated the use of standardized 
testing for assessment and placement of entering community college students. The New 
Jersey College Basic Skills Placement Test (NJCBSPT) was adopted for a system wide, three- 
semester pilot beginning in spring 1989 (Board of Trustees, 1988), 

The test is designed to measure reading writing and mathematics skills of students 
entering college. Each of the five sections-Essay, Reading Comprehension, Sentence Sense, 



Presented at the 33rd Annual Forum of the Association tor Institutional Research, Chicago 
Q IL, May 19, 1993. 

ERIC 3 



Placement Testing and Student Success 



Page 2 



Mathematical Computation, and Elementary Algebra-contains multiple choice questions, 
with the exception of the essay. Because these are proficiency tests, they assist college 
personnel in determining whether an incoming student will profit more from placement in 
college-level courses than from placement in any of the pre-college courses offered by the 
college (Basic Skills Council, 1984). 

The testing program was partially implemented in Fall 1989 at (then) South Central 
Community College (SCCC). All degree-seeking, first-time students must take the test, as 
well as students who have attended the college previously and have accumulated 12 credits 
and all students who wish to enroll in EngUsh and/or mathematics courses. Students are 
tested in computation, algebra (optional), reading, essay, and sentence. Scaled scores in 
these tests are used as the basis for placement recommendations. 

It is a college policy that a student may be re-placed in a higher level course (above 
the recommended level) based on performance on an in-class, locally-developed test given by 
the instructor during the first wee.k of class or as a result of an appeal process which takes 
into account additional information, such as evidence of appropriate instruction since the test 
was administered. Thus, students who enroll at a higher levels than recommended do not 
represent a random sample, nor are they circumventing the placement process. Howevpr, for 
purposes of this study, their performance is analyzed based on the NJCBSPT 
recommendation. 

The objectives of this study are to determine if: (1) students who are placed in 
courses consistent with their competence in basic skills, as measured by the NJCBSPT, have 
a higher rate of success in those recommended courses than those who enroll initially in 
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higher level courses and (2) students who are successful in basic skills courses enroll in 
greater percentages than those who do not follow the recommendation, in college-level 
mathematics and English, and complete them successfully. The underlying assumption is that 
the content of the recommended courses are appropriate for the skills being tested. In all 
instances, recommendations are derived from the NJCBSPT score. 

This study is the beginning of a longitudinal tracking project that follows students 
from basic skills testing through program completion. Since data for 1989 were incomplete, 
the initial cohort for this study comprises those students who tested and enrolled at SCCC in 
fall 1990. This first report covers the first six semesters of enrollment (fall 1990 through 
spring 1993 for this cohdrt) and examines the placement scores and recommendations, the 
courses in which the student actually enrolled, student success in those courses, the impact 
on continued enrollment at the college and success in higher level courses. The data are also 
used to address the question of whether the cut-off scores were established and applied 
accurately with respect to course content or, conversely, if the course content for the 
recommended courses is consistent with the skills determined by the test. 
Review of the Literature 

Other studies have been carried out that examined basic skills and academic 
performance. One of the earliest studies at Queensborough Community College (CUNY) in 
1974 compared the success in college courses of students from the same entering class 
enrolled in basic skills courses with those not enrolled in these courses. However, this 1974 
study by Bergman and Gerace did not analyze the cohort of students recommended for basic 
skills courses and whether they enrolled in those courses; the comparison was between those 
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who needed basic skills with those who did not. While the results (predictably) showed a 
higher percentage of A and B grades by non-basic skills students, there was a four percent 
increase in passing grades by basic skills students over two years and a five percent decrease 
in F grades. 

Studies at Miami-Dade Community College (Belcher, 1984 ; Losak, 1982; Losak & 
Morris, 1982) examined, respectively, entering basic skills and CLAST performance, 
retention based on basic skills assessment performance and academic progress as related to 
basic skills. Belcher's study showed that "students above the cut score in all three basic skill 
areas were more likely to re-enroll and graduate than students who scored below (the cut) on 
all three criteria. Like the Queensborough study above, this study compared students who 
needed basic skills with those who did not. Her study also showed that students who enter 
with deficiencies are more likely to exit with deficiencies, either by withdrawing from the 
course (or college) or failing the exit test. Losak' s retention study also compared students 
who required basic skills courses based on their placement cut-off score and those who did 
not need basic skills work. The group of entering students who scored above the cut-off on 
all three placement tests and required no remediation showed a 65.6 percent return rate the 
following term. Those who scored below the cut-off on all three tests and needed the 
remedial courses showed a 48.7 percent return rate. The study also reported that there was 
no difference in the return rate if the student passed one or two of the tests. He concluded 
that if a student scores below the cutoff on any (placement) test, the probability for attrition 
increases dramatically when compared to students who passed all (the tests). The academic 
progress study by Losak and Morris also looked at ethnic categories but concluded overall 
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that (1) retention rates were higher for students above the placement test score cut-off, (2) a 
lower percentage of students below the cut-off made satisfactory performance and (3) a lower 
percentage of students below the cut-off showed continuous enrollment. 

A 1985 study by Oromaner at Hudson County (NJ) Community College showed that, 
over five semesters, students in need of no basic skills were more likely to return to college. 
Further, students in need of basic skills were more likely to leave after one semester; 73 
percent are gone after three semesters. Those students not needing remediation averaged 
four semesters of attendance and 60 percent were gone after three semesters. 

Haase and Caffrey (1983) at Sacramento City College (CA) found a direct correlation 
between assessment scores and attained grades: the student reading below the 10th grade 
level has very little chance of receiving a C grade or better in any class except those created 
to meet the learning skill need. Further, the retention rates in English composition classes 
reflect the efficacy of correct placement in those classes under the assessment/placement 
process. Seven years later, also at Sacramento City College, Boese (1990) found that 
placement test scores and their subsequent recommendations for courses were significant 
predictors of QPA but not persistence. Hughes and Nelson (1991), using discriminant 
analysis to determine effect of placement scores on success, found "a very low degree of 
separation between students who receive passing grades and those who do not receive passing 
grades." Boggs (1984) at Butte College (CA) presented a case for basic skills intervention. / 
His study, analyzing basic skills assessment on student achievement and persistence, found 

(1) that the assessment program, per se, did not effect withdrawal rates in writing courses, 

(2) that students who were enrolled in developmental English based on results of a writing 

V 
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sample did not get significantly higher grades in freshman composition courses, (3) that 
students who were enrolled in developmental English using the Stanford TASK instrument 
received significantly higher grades in the freshman composition course, and (4) that students 
who were enrolled in developmental English showed a higher, but not statistically significant, 
persistence in freshman composition courses. This last study shows that taking the 
developmental courses when recommended by a standardized placement test can be predictive 
of persistence. 
Data Used in Study 

Data used for these analyses include 1) test data (raw and scaled scores for the 
computation, algebra, es^ay, and sentence sense tests) directly from the test scan files; 2) 
recommendations calculated from test score consistent with SCCC's cutoff and practices; 3) 
course data (lowest and highest level courses taken in Math and English with respective 
grades), and 4) academic data (QPA and number of terms attended) from the college's 
student records system. Mathematics and English were selected for study because college- 
level courses in those subjects are required for completion of degree programs. The cohort 
comprises all students (N=804) who took the placement test between March 1 and October 
1, 1990 and enrolled for the first time in fall 1990. [Totals for the different test may not 
match because all students do not have to take all tests and/or there may be erroneous Social 
Security Numbers.] Data analysis was done using SAS V6.06. 
Test Scores - Requirements for Cut-off Scores 

Table 1 summarizes South Central Community College's cutoff scores and subsequent 
course recommendations. Placement in a mathematics course is based on both the 
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computation and algebra tests. Students scoring 160 or below on the computation test are 
recommended for enrollment in Basic Math I. If a student scores above 160 in computation, 
the score on the algebra test is used to determine placement in Basic Math II or a college- 
level mathematics course. Placement in English is based on a combination of the essay and 
sentence sense tests. Students who score above the cutoff on the essay test are recommended 
for college-level EngHsh. The sentence test score is used as further placement assistance 
when the essay score is near the cutoff. 



Table 1. Course Level Recommendations Based on the SCCC Cut-off and 
NJCBSPT Scores 



Mathematics 


English 


Ranges 


ReconmiendatioD 


Ranges 




Recommendation 


< = 160 in 
Computation 


Basic Math I 


< 6 in Essay 
and < 160 in 
Sentence 


Basic English 


> = 161 in 
Computation 
and 

< = 171 in Algebra 


Basic Math II 


6 in Essay 
and > = 160 in 
Sentence 


English 
Composition 


> = 172 in Algebra 


College Math 


> = 7 In Essay 


English Comp 



Test Scores - Results and Recommendations 

Table 2a presents the course recommendations based on the scaled scores for the 
computation and algebra tests. Of the 702 students who took the NJCBSPT computation test 
in 1990 and enrolled at South Central in Fall 1990, 499 (71%) scored 160 or less, with a 
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mean scaled score of 151.8 and were recommended for Basic Math I. One hundred seventy- 
nine students were recommended for Basic Math n with a mean scaled score of 167.1 on the 
computation test. Fifty-three of these students did not attempt the algebra test; their 
placement was based only on their computation score. Twenty-four were recommended for a 
college-level mathematics course with a mean algebra score of 179.7. 



Table 2a. Math Placement Tests: Recommendations and Scores 



Recommended Course 


N 


% 


Mean 


Standard 








Score 


Deviation 


Basic MatU I 


499 


71.1 


151.8 


5.06 


Basic Math U 


179 


25.5 


167.2 


5.28 


College Math 


24 


3.4 


179.7 


4.71 



Table 2b shows the comparable data for the English placement tests: Of the 673 
students who took the NJCBSPT essay and sentence tests in 1990 and enrolled at South 
Central. 306 (45.5%) were recommended for Basic English and 367 (54.5%) were 
recommended for English Composition. 
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Recommended 
Course 


Test 


N 


% 


Mean 
Score 


Standard 
Deviation 


Basic English 


Essay 


301 


45.1 


5.5 


1.23 




Sentence 


306 


45.5 


150.6 


8.61 


English 


Essay 


365 


54.9 


7.7 


1.08 


Composition 


Sentence 


367 


54.5 


165.6 


7.68 



Recommended vs. Enrolled 



As illustrated in Table 3, there is some discrepancy between course recommendations 
and course enrollment. More than 60 percent of those students recommended for either 
Basic Math I or Basic English enrolled in those respective courses; less than half of the 
students recommended for Basic Math II actually enrolled in that course, with 13 percent 
opting to take the lower level Basic Math I. As was mentioned above, students may be "re- 
placed" in a higher level course based on an in-class assessment (not a retest of NJCBSPT) 
or if the student demonstrates additional instaiction since the NJCBSPT was administered: 
only seven percent of the Basic Math I students and eight percent of the Basic Math II 
students fell into this category. However, almost 20 percent of the Basic English students 
enrolled in English Composition instead of the recommended course 

A curious statistic is that, after six semesters, a sizable percentage of students had not 
enrolled in any mathematics or English course. A later analysis of the data will examine the 
relationship between courses taken and withdrawal from the college. 




Presented at the 33rd Annual Forum ot the Association for Institutional Research Chi-aso 
IL, May 19, 1993. ' ^ ' 

u 



Placement Testing and Student Success 



Page 10 



Table 3. Course Recommendations vs. Actual Levels of Enrollment 



Recommended 


Lower Level 


ReconMnended 
Level 


Higher Level 


No Math/Engl 
Course 


N 


% 


N 


% 


N 


% 


N 


% 


Basic Math I 






322 


64.5 


35 


7.0 


142 


28.5 


Basic Math H 


24 


13.4 


82 


45.8 


15 


8.4 


58 


32.4 


College Math 






19 


79.2 






5 


20.8' 


Basic English 






189 


61.7 


57 


18.6 


60 


19.6 


English Comp. 


30 


8.2 


264 


71.9 






73 


19.9 



Effect of Placement Scares on Success at Recommended Level 

The rate of successful completion was compared for each recommended course and 
level at which the student enrolled (see Table 4). For this analysis, grades were 
dichotomized to reflect successful completion (A, B, C) or non-successful completion (D, F, 
W, or No Grade). Of the students who were recommended for Basic Math 1 and enrolled in 
Basic Math I, 63 percent completed successfully; 78 percent of those who were 
recommended for Basic Math II successfully completed the course. This is contrasted with a 
60 percent success rate for those who were recommended for Basic Math 1 as a result of the 
NJCBSPT but took the higher level course (as explained above). It should be noted that 
there was a 14 point difference in mean scores--as well as the 18 percentage points in success 
rates-between the two groups, and the average mean score of the "re-placed" group was 
seven points lower than the cut-off for Basic Math II. 
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Similar results are noted for English. Seventy-two percent of those students 
were recommended for Basic English and took that course completed it successfully. Eighty 
percent of those recommended for and took English Composition completed successfully. 
However, only 68 percent of those who were recommended for Basic English (as a result of 
the NJCBSPT) but took English Composition completed successfully. Again, there was a 15 
point difference between the mean scores of those recommended for English Composition 
based on NJCBSPT and those who were "re-placed" in English Composition; and the average 
mean score on the sentence section for the "re-placed" group was more than eight points 
lower than the recommended cut-off for English Composition. Consequentiy, students who 
should be in Basic Math II as a result of the NJCBSPT had a higher success rate than those 
who did not follow the original recommendation and went into that course anyway. This 
conclusion holds for English Composition as well. 



Table 4. 



ERIC 



The Relationships Among Recommended Courses, Course 
Enrollment Level and Successful Completion 



Recommended 
Course 


Level of Enrollment 




Low 


er 


Recommended 


Higher 


xN 


Pet. 
Succ 


Mean 
Score 


N 


Pet. 
Succ 


Mean 
Score 


N 


Pet. 
Succ 


Mean 
Score 


Basic Math I 








322 


63 


151.7 


35 


60 


153.2 


Basic Math II 


24 


92 


161.4 


82 


78 


167.2 


15 


67 


170.6 


College Math 


0 


n/a 




19 


84 


179.7 








Basic English 








189 


72 


147.9 


57 


68 


152.5 


English Comp 


30 


71 


164.6 


264 


80 


167.7 
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NOTE: Mean scores in Basic Math I and II are for the computation test; the mean score for 
College Matli is for the algebra test. Mean scores for both Basic English and English 
Composition are for the sentence sense test. 

An analysis of variance was used to determine if there was a difference between the 
mean test scores of the group of students who were successful in courses and the students 
who were unsuccessful in the same courses. For all analyses in this study, a minimal level 
of significance was established at p > .01. The only category showing a statistically 
significant difference in the means was in the group which was recommended for Basic Math 
I and enrolled in that course (see Table 5). While that difference was statistically significant, 
it is open to further investigation whether the three-point difference between the mean scores 
of the two groups would indicate an academically significant difference in skill level and, 
therefore, necessitate changes in present cutoff scores. [Referring to Table 2a, the mean 
score of all students recommended for Basic Math I was 151.8.] 



Table 5. Successful Completion in Enrolled Courses Based 
on Course Recommendations 



Recommended 
Course 


Enrolled 
Course 


Succeeded 


Did Not 
Succeed 


N 


Mean 
Score 


N 


Mean 
Score 


Basic Math I 


Basic Math I 


203 


152.8 


119 


149.8 


Basic Math I 


Basic Math II 


16 


154.4 


19 


151.3 


Basic Math II 


Basic Math II 


65 


158.7 


17 


158.2 


Basic English 


Basic English 


136 


148.2 


53 


147.5 


Basic English 


English Composition 


39 


152.4 


18 


153.1 


NOTE: Mean scores in Basic Math 1 and II are for t 


he computation test; the mean 
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Effect of EnroUment in Recommended Courses on Higher Level Courses and Persistence 

Based on an analysis of enrollment between fall 1990 and spring 1993, and compared 
with the highest level mathematics course taken (See Tables 6a and 6b), almost half the 
students recommended for and enrolled in Basic Math I show Basic Math I as their highest 
level math course, and 70 percent of these students had withdrawn from the college; 25 
percent show Basic Math II as their highest level mathematics course, and 65 percent of 
these students had withdrawn from the college. However, just under 30 percent of those 
students originally recommended for Basic Math I enrolled in a college level mathematics 
course, and only 25 percent of these students had withdrawn from the college over the six 
semesters. [Cf Table 3:* One hundred twenty-seven students recommended for Basic Math I 
(25.5 percent) never enrolled in any mathematics course and withdrew from the college; 15 
students are still enrolled but have not as yet taken a mathematics course.] 

Of the students recommended for and enrolling in Basic Math II, 45 percent show 
Basic Math II as their highest level mathematics course, with 70 percent having withdrawn: 
the remaining 55 percent showed a college-level mathematics course on their records and 
only 31 percent had withdrawn from the college. [Cf Table 3: Forty-five of those 
recommended for Basic Math II (7.3 percent) never enrolled in any mathemxatics course and 
withdrew from the college; 13 are still enrolled but have not enrolled in a mathematics 
course.] 
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Table 6a. Highest Level Mathematics Course Taken Before 
Withdrawing from the College 



Enrolled in 
Recommended 
Couree 


Highest Course Taken 


Basic Math I 


Basic Math H 


College Math 


Enrolled 


W/D 


Enrolled 


W/D 


Enrolled 


W/D 


Basic Math I 

(N= 322) 


N 


151 


105 


80 


52 


91 


23 


% 


46.9 


69.5 


24.8 


65.0 


28.3 


25.3 


Basic Math H 

(N=82) 


N 






37 


26 


45 


14 


% 






45.1 


70.3 


54.9 


31.1 



Of the 189 studeijts recommended and enrolled in Basic English, 42 percent showed 
this course as their highest English course, but 83.8 percent had withdrawn from the college, 
Almost 60 percent subsequently enrolled in English Composition with only 46.8 percent of 
them withdrawing from the college. [Fifty-seven students recommended for Basic English 
(18.6 percent) did not take any English course and withdrew from the college; three are still 
enrolled but have not taken an English course.] 



Table 6b. Highest Level English Course Taken Before 
Withdrawing from the College 



Recommended 
and 
Enrolled 


Basic 
English 


English 
Composition 


Enrolled 


W/D 


Enrolled 


W/D 


Basic English 

(N=189) 


N 


80 


67 


109 


51 


% 


42.3 


83.8 


57.7 


46.8 
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Among all students who took the NJCBSPT prior to enrolling in fall 1990 (804), only 
23.8 percent (191) were still enrolled in spring 1993. For the group recommended for Basic 
Math I--again, regardless of highest level of enroUment-the six-semester retention rate is 
44.1 percent; for the Basic Math II group it is 51.3 percent. The Basic English group has a 
37.6 percent retention rate. [No cross-check was performed to ascertain how many students 
are in both a mathematics and the English groups.] These statistics should be used in the 
assessment process to evaluate the relationship between student skill level and course content. 
Relation of Success in First Basic-Level Course to Persistence and QPA 

Students were categorized into five groups for both Math and English and then 
compared by mean termS of attendance and Quality Point Average (see Table 7). Analyses 
of variance of the mean overall QPAs and mean terms of attendance for the five categories 
indicate that those means are statistically different for both English and Math groups. 
Summary and Recommendations 

The basic goal of this study was to determine if students who enrolled in courses that 
were recommended on the basis of basic skills placement testing were more or less successful 
than those who enrolled in higher or lower level courses. This study used data from one 
college for the entering group for one term. These results should not be generalized for 
other colleges, other years at SCCC or the NJCBSPT. This success was measured in four 
ways: (1) success in the basic skills course, (2) completion of Math and English courses 
applicable to the associate degree or certificate, (3) overall academic achievement (QPA), 
and (4) continued enrollment (persistence). 
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Table 7. Persistence and QPA Based on NJCBSPT Recommendations 



Category 

(includes only students 
recommended for 
Basic Level Courses) 


Mathematics 


English 


N 


Mean 
QPA 


Mean # 
of Terms 
Attended 


N 

J. ^ 


QPA 


iVlcaU n 

of Terms 
Attended 


Recommended & 
Successful 


268 


2.71 


4.15 


136 


2.25 


3.84 


Reconmiended & 
Not Successful 


136 


1.45 


3.63 


53 


.99 


2.75 


Higher Course & 
Successful 


31 


2.92 


4.74 


39 


2.63 


3.92 


Higher Course & 
Not Successful 


19 


2.33 


3.58 


18 


1.84 


3.78 


No Math Course 
No English Course 


224 


2.07 


2.47 


60 


1.56 


1.70 



Initial results show that a higher percentage of students who followed the 



recommendations of the NJCBSPT were successful, compared to those who did not: 63 
percent of the students who were recommended for Basic Math I were successful at that 
level. Seventy-nine percent (79.3%) of those recommended for Basic Math II were 
successful. Seventy-two percent (72.0%) of those recommended for Basic English were 
successful. Correspondingly, 60 percent of those who were recommended for Basic Math I 
but took Basic Math II were successful, 67 percent of those who were recommended for 
Basic Math II but who took a college-level mathematics course were successful and 68 
percent of those who were recommended for Basic English but enrolled in English 
Composition were successful. 
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Data for completion of college-level courses are inconclusive: 28 percent of the 
students who were recommended for Basic Math I and enrolled at that level have completed 
a college-level math course, as have 54.8 percent of the students recommended for and 
enrolled in Basic Math II and 57.7 percent of the students recommended for and enrolled in 
Basic English. This evidence warrants an analysis of reasons for withdrawal or, for those 
still enrolled, why they did not register. 

Obviously, students who were successful in their recommended basic-level courses 
had significantly higher QPAs than those who were unsuccessful in either recommended or 
higher level courses or who did not enroll in either Math or English courses. Since the data 
show that non-successfur students do not withdraw immediately, further research is necessary 
to determine if they either re-enrolled for the course or tried other courses before 
withdrawing. 

Students who were successful in their recommended basic-level courses tended to 
continue enrollment at SCCC for a slightly greater number of terms. The successful 
completion of pre-college (basic skills) courses are highly related to QPA, but have much 
less effect on persistence. This is consistent with Boese (1990) who reported that 
recommendations for courses were significant predictors of QPA but not persistence. It 
should also be noted, however, that the retention or persistence rates for students who went 
on after following the NJCBSPT recommendation were higher than for the cohort as a 
whoL, 

Withdrawal rates are presented regardless of success in the mathematics or English 
courses. However, the pattern of the highest percentage withdrawal from the lowest level 
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courses, regardless of recommendatioii, is worthy of further investigation. It may be that 
other factors are intervening. 

In terms of assessment, the scores and recommendations for Basic Math I, in light of 
the relatively low 63 percent successful completion rate-compared to Basic Math II, College 
Math, Basic English, or English Composition-should be reviewed to ascertain (1) if, because 
this is the lowest L^.vel mathematics course, students who have a skill level too low to benefit 
from the current course content are being included and/or (2) the course content does not 
reflect the skills being tested. Since this is the first point in a longitudinal study, additional 
analyses are need to support these hypotheses. 

This study establishes a model and demonstrates a need for additional and/or 
expanded research. The small and uneven cell sizes in some of the analyses indicates the 
need for the use of subsequent years' data; however, the initial results do indicate the need 
for additional consideration. As part of the continuing assessment process, the relationship 
of scores, recommendations and course content should be reviewed. 



V 
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